Gamified Exploratory GUI Testing of Web Applications: a Preliminary Evaluation

In the context of Software Engineering, testing is a well-known phase that plays a critical role, as is needed to ensure that the designed and produced code provides the expected results, avoiding faults and crashes. Exploratory GUI testing allows the tester to manually define test cases by directly interacting with the user interface of the finite system. However, testers often loosely perform exploratory GUI testing, as they perceive it as a time-consuming, repetitive and unappealing activity. We defined a gamified framework for GUI testing to address this issue, which we developed and integrated into the Augmented testing tool, Scout. Gamification is perceived as a means to enhance the performance of human testers by stimulating competition and encouraging them to achieve better results in terms of both efficiency and effectiveness. We performed a preliminary evaluation of the gamification layer with a small sample of testers to assess the benefits of the technique compared with the standard version of the same tool. Test sequences defined with the gamified tool achieved higher coverage (i.e., higher efficiency) and a slightly higher percentage of bugs found. The user's opinion was almost unanimously in favor of the gamified version of the tool

A Review on Tools, Mechanics, Benefits, and Challenges of Gamified Software Testing

Gamification is an established practice in Software Engineering to increase effectiveness and engagement in many practices. This manuscript provides a characterisation of the application of gamification to the Software Testing area. Such practice in fact reportedly suffers from low engagement by both personnel in industrial contexts and learners in educational contexts. Our goal is to identify the application areas and utilised gamified techniques and mechanics, the provided benefits and drawbacks, as well as the open challenges in the field. To this purpose, we conducted a Multivocal Literature Review to identify white and grey literature sources addressing gamified software testing. We analysed 73 contributions and summarised the most common gamified mechanics, concepts, tools and domains where they are mostly applied. We conclude that gamification in software testing is mostly applied to the test creation phase with simple white-box unit or mutation testing tools, and is mostly used to foster good behaviours by promoting the testers’ accomplishment. Key research areas and main challenges in the field are: careful design of tailored gamified mechanics for specific testing techniques; the need for technological improvements to enable crowdsourcing, cooperation, and concurrency; the necessity for empirical and large-scale evaluation of the benefits delivered by gamification mechanics

A Metric Framework for the Gamification of Web and Mobile GUI Testing

System testing through the Graphical User Interface (GUI) is a valuable form of Verification & Validation for modern applications, especially in graphically-intensive domains like web and mobile applications. However, the practice is often overlooked by developers mostly because of its costly nature and the absence of immediate feedback about the quality of test sequence. This paper describes a proposal for the Gamification of exploratory GUI testing. We define - in a tool and domain- agnostic way - the basic concepts, a set of metrics, a scoring scheme and visual feedbacks to enable a gamified approach to the practice; we finally discuss the potential implications and envision a roadmap for the evaluation of the approach

Is ChatGPT Capable of Crafting Gamification Strategies for Software Engineering Tasks?

Gamification has gained significant attention in the last decade for its potential to enhance engagement and motivation in various domains. During the last year ChatGPT, a state-of-the-art large language model has received even more attention both in the field of scientific research and in common use by individuals or companies. In this study, we investigate the possibility of adopting ChatGPT as a tool for designing gamification platforms in the Software Engineering domain. Leveraging the capabilities of ChatGPT, we assess how good is it at generating effective suggestions and ideas for designers or developers. To evaluate ChatGPT's potential as a gamification platform creator we narrowed the context to one particular Software Engineering activity, asking for possible aspects of the activity to be gamified. Each proposed aspect was subsequently unraveled by ChatGPT both asking in a shared and separate context, first following the conversational nature of the model, then applying a validated design framework. The study assesses ChatGPT's ability to select and integrate game elements to build a thriving gamification environment by framing the design of the platform to a state-of-the-art conceptual framework. To evaluate the goodness of the design choices made we relied both on the Octalysis framework and on personal experience. The findings of the papers show that ChatGPT can only create simple playful experiences not very effective. Although, by instructing the model with more specific desired mechanics and dynamics, it is possible to guide it toward the application of the ideas suggested. We argue that ChatGPT is not capable of building a gamified environment on its own, but it could still be used to build the foundation of a gamification platform as long as the designers refine and rough out the advice gained from a user-centered solution

GERRY: A Gamified Browser Tool for GUI Testing

Graphical User Interface (GUI) testing is a relevant step of the soft- ware development process which is not often performed thoroughly due to its unappealing nature, to the inherent fragility of test cases, and to the fact that test cases – composed of long and complicated sequences of operations – have to be manually written by testers. We propose GERRY , a Capture & Replay GUI testing tool which implements an approach based on Gamification, i.e., the application of gaming elements to non-ludic activities. The purpose of the tool is to increase the engagement of the testers when performing GUI test case definition tasks. The tool makes use of mechanics typical of games such as progress indicators, leaderboards, and unlockable rewards, to increase user interest and involvement. GERRY also generates reports (i.e., traces of all actions and milestones reached during a session), written logs of the performed testing sessions, and scripts compatible with existing GUI testing tools (SikuliX and Selenium) for replay purposes

An analysis of widget layout attributes to support Android GUI-based testing

In the context of GUI testing, identifying robust locators (i.e., attributes to unambiguously identify on-screen widgets to be used in test sequences) is still considered an unsolved challenge by the researchers' community. The frequent variation of attributes between different releases of the System Under Test (SUT) leads in fact to testing fragility, i.e., test case failing because of invalidated locators. Recent studies have highlighted the benefits of adopting multi-locator approach, i.e., the combination of multiple locators to enhance the robustness of widget identification. The objective of this work is to provide insights into the composition of Android applications, assessing the characteristics of different layout-based properties and their suitability to be used as locators for widgets in the context of GUI-based testing. We investigated the state of the practice by analysing the distribution of widget values within 30 real apps selected from the Google Play Store. For those apps, we selected two different versions to examine how they evolved over time from both visual and structural perspectives. The results of our analysis showed that missing values, variability, and instability of attributes make it hardly possible to identify a single attribute or technique (either coordinate-, property-, or visual-based) capable of providing robust GUI testing over multiple releases of mobile SUTs

Survival of the Tested: Gamified Unit Testing Inspired by Battle Royale

While testing takes on a fundamental role to verify software quality and correctness, it often results to be overlooked in the educational field and students often approach it unwillingly, due to its repetitiveness. Our aim is to exploit gamification to engage students by providing them with dynamics like competition, self-expression, and personal improvement. We designed and developed Unit Brawl, a gamified application meant to manage multiple rounds, each one consisting of students developing Java programs and unit tests to be executed on each other. The players collect points by writing correct code that does not make the other players’ test cases fail, or by writing test cases capable of detecting defects in the other players’ code. The results of a preliminary evaluation to assess the functionality and performance of Unit Brawl look promising. They make us confident about its stability, so we plan an evaluation with students in order to verify the effectiveness of the applied game elements in enhancing the students’ interest towards testing topics and their learning

Guidelines for GUI Testing Maintenance: A Linter for Test Smell Detection

Context: GUI Test suites suffer from high fragility, in fact modifications or redesigns of the user interface are commonly frequent and often invalidate the tests. This leads, for both DOM- and visual-based techniques, to frequent need for careful maintenance of test suites, which can be expensive and time-consuming. Objective: The goal of this work is to present a set of guidelines to write cleaner and more robust test code, reducing the cost of maintenance and producing more understandable code. Based on the provided recommendations, a static test suite analyzer and code linter has been developed. Method: An ad-hoc grey literature research was conducted on the state of the practice, by performing a semi-systematic literature review. Authors’ experience was coded into a set of recommendations, by applying the grounded theory methodology. Based on these results, we developed a linter in the form of a plugin for Visual Studio Code, implementing 17 of the provided guidelines. The plugin highlights test smells in the Java and Javascript languages. Finally, we conducted a preliminary validation of the tool against test suites from real GitHub projects. Conclusions: The preliminary evaluation, meant to be an attempt of application of the plugin to real test suites, detected three main smells, namely the usage of global variables, the lack of adoption of the Page Object design pattern, and the usage of fragile locator such as the XPath

On Effectiveness and efficiency of Gamified Exploratory GUI Testing

Context: Gamification appears to improve enjoyment and quality of execution of software engineering activities, including software testing. Though commonly employed in industry, manual exploratory testing of web application GUIs was proven to be mundane and expensive. Gamification applied to that kind of testing activity has the potential to overcome its limitations, though no empirical research has explored this area yet. Goal: Collect preliminary insights on how gamification, when performed by novice testers, affects the effectiveness, efficiency, test case realism, and user experience in exploratory testing of web applications. Method: Common gamification features augment an existing exploratory testing tool: Final Score with Leaderboard, Injected Bugs, Progress Bar, and Exploration Highlights. The original tool and the gamified version are then compared in an experiment involving 144 participants. User experience is elicited using the Technology Acceptance Model (TAM) questionnaire instrument. Results: Statistical analysis identified several significant differences for metrics that represent the effectiveness and efficiency of tests showing an improvement in coverage when they were developed with gamification. Additionally, user experience is improved with gamification. Conclusions: Gamification of exploratory testing has a tangible effect on how testers create test cases for web applications. While the results are mixed, the effects are most beneficial and interesting and warrant more research in the Further research shall be aimed at confirming the presented results in the context of state-of-the-art testing tools and real-world development environments

A Framework for the Gamification of GUI Testing

Software testing is a critical activity in the software development process. Several techniques have been proposed, addressing different levels of granularity from low-level unit testing to higher-level exploratory testing through the software’s graphical user interface (GUI). In modern software development, most test cases are obtained by automated test generation. However, while automation generally achieves high coverage in code-level white-box testing, it does not always generate realistic sequences of interactions with the GUI. By contrast, manual exploratory testing has survived as a costly, error-prone, and tedious yet crucial activity. Gamification is seen as an opportunity to increase user satisfaction and engagement while performing testing activities. It could also enable and encourage crowdsourced testing tasks. The purpose of the study described in this chapter is to provide a framework of gamification mechanics and dynamics that can be applied to the practice of manual exploratory GUI testing. We provide an implementation of the framework as an extension of an existing manual exploratory GUI testing for Web applications, and we provide a preliminary evaluation of the gamified tool in terms of provided efficiency, effectiveness, and user experience. Our results show that the gamified solution makes the testers obtain test suites with higher coverage while reducing slightly the number of bugs signalled while traversing the applications under test. The gamified tool also was considered to provide a positive user experience, and the majority of participants expressed their willingness to use such instruments again in the future. As future work, we foresee the implementation of the framework in a stand-alone tool and in-depth empirical experiment to evaluate quantitatively the benefits and drawbacks provided by such mechanics in real testing scenarios